Papers with Short Answer Scoring
Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation (2020.acl-srw)
Copied to clipboard
Hiroaki Funayama, Shota Sasaki, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Masato Mita, Kentaro Inui
| Challenge: | Recent Short Answer Scoring systems use Quadratic Weighted Kappa (QWK) but it is unsatisfactory when measuring their effectiveness in actual usage. |
| Approach: | They propose a task formulation of Short Answer Scoring (SAS) that matches actual usage and extracts as many scoring predictions that are not critical scoring errors (CSEs). |
| Outcome: | The proposed system predicts scores with zero critical scoring errors (CSEs) for 50% of test data at maximum by filtering out low-reliability predictions on the basis of a certain confidence estimation. |
Semi-Supervised Clustering for Short Answer Scoring (L18-1)
Copied to clipboard
| Challenge: | Existing approaches to SAS use unsupervised clustering and have teachers label some items after clustering. |
| Approach: | They propose to use semi-supervised clustering to provide structured groups of answers in addition to a score. |
| Outcome: | The proposed method improves clustering performance from 0.504 kappa for unsupervised clustering to 0.566 kppa. |